Video compression method

ABSTRACT

A method of decoding and encoding video, the encoding method comprising: receiving a video; performing a shrink operation on the received video, the shrink operation comprising: creating a first lower resolution video from the video; and sequentially creating additional N−1 lower resolution videos, each one of the additional lower resolution videos created from the preceding lower resolution video; compressing the lowest resolution video; creating a lowest resolution reconstructed video by decompressing the lowest resolution compressed video; performing a first raise operation on the lowest resolution reconstructed video, the first raise operation comprising sequentially creating N higher resolutions reconstructed videos, each one of the higher resolution reconstructed videos created from the preceding lower resolution reconstructed video by: creating a higher resolution video from the lower resolution reconstructed video; computing a residual between the respective lower resolution video and the created higher resolution video; compressing the computed residual; decompressing the compressed residual; and combining the decompressed residual with the created higher resolution video, yielding the respective higher resolution reconstructed video; and providing a bit stream comprising the lowest resolution compressed video, said compressed residuals and control information.

CROSS-REFERENCES TO RELATED APPLICATIONS

This patent application claims priority from and is related to U.S.Provisional Patent Application Ser. o. 61/710,723, filed Oct. 7, 2012,this U.S. Provisional Patent Application incorporated by reference inits entirety herein.

TECHNOLOGY FIELD

The present invention is in the field of video compression.

BACKGROUND OF THE INVENTION

Raw video files consume huge amounts of space. For example, a HighDefinition(HD) movie with 60 frames per second(fps), frame resolution of1920×1080 pixels, color depth 3, and 8 bits per color, consumes:

-   -   1,920*1,080*60*3=373,248,000 Bytes per second.

And two hours of such movie would consume:

373,248,000*7,200=2,687,385,600,000 Bytes 3 Tera bytes(Tbytes).

To store the movie on regular DVD disks, i.e. 4.7 Giga bytes(Gbytes)disks, we need:

-   -   ≈2,687/4.7≈600 DVD disks.

And to transmit the movie over the Internet, say over fast 100 Mbpschannels, we need:

≈2,687,386*8/100≈2,149,908 seconds 60 hours.

Video compression is the art of reducing the video size withoutaffecting the perceived quality.

Video content is not always taken with the best equipment and the bestphoto shooters. In such cases digital image processing, also known asvideo enhancement, can substantially improve the visible quality of thevideo, and help the video compression process. Some of the more knownmethods for video enhancements use video preprocessing tools such as thefollowing:

De-interlacing Interlaced movie can be problematic when recording fastmoving objects. The moving object can be in one place in the “even”picture, and in another place in the “odd” one, yielding a “stripped”picture which is very disturbing.

De-blocking Block-like artifacts are the side effect of the currentMPEG's low-quality highly compressed videos. De-blocking greatlyincreases the quality of such videos.

Sharpening emphasizes texture and detail, and is critical whenpost-processing most digital images. An “unsharp mask” is actually usedto sharpen an image.

De-noising Some degree of noise is always present in any electronicdevice that transmits or receives a “signal”. For television this signalis the broadcast data transmitted over cable or received at the antenna;for digital cameras, the signal is the light which hits the camerasensor. Video De-noising is the process of removing noise from a videosignal.

Stabilization is used to reduce blurring associated with the motion ofcamera. Specifically, it compensates for pan and tilt of a camera orother imaging devices. With video cameras, camera shake causes visibleframe-to-frame jitter in the recorded video.

Camera Calibration is important in order to get stable and reliableimages. Cameras that operate out of sync or are imprecisely adjusted cancreate blurry or confused images.

Such tools can greatly improve the video quality and help thecompression process. Here, we ignore this issue and assumes that thevideo has already been preprocessed as required. See Ref. [1] for moredetails.

A digital video consists of multiple streams such as video, audio, andcontrol, that are stored together in the same container file. Forexample, common containers formats are: AVI (Audio Video Interlaced),WMV (Windows Media Video), FLV (Flash Video), MOV (Apple Quick TimeMovie). The video stream itself is usually independent of the otherstreams, or of the container type, and can be represented in manydifferent formats. A media player, such as Apple iTunes, and MicrosoftWindows Media Player, displays the video on the screen, using thecorresponding Codec (Encoder/Decoder) software.

The displayed video is usually represented in the raw RGB color spaceformat because the human visual system works in a similar way, i.e., thehuman eye color vision is based on red, green and blue color sensors.The raw RGB file 100 is schematically depicted in FIG. 1, comprising aheader section 120 followed by frames 130. The Header 120 contains thevideo parameters such as: n—number of rows, m—number of columns, andN—number of frames. A frame 130 contains n*m pixel values, each atriplet for the R, G and B.

The raw YUV color space format is another very useful format for videorepresentation. Here, Y corresponds to the black and whiterepresentation of the video, and U and V to the added color differences.There are many similar formulas for converting RGB to YUV and viceversa. One of them, see Ref. [2], is exemplified in FIG. 2, where theRGB to YUV transforming formula is given in unit 210, and the YUV to RGBtransforming formula is given in unit 220. The raw YUV file 300 isschematically depicted in FIG. 3, comprising a header section 310 as inunit 120 of FIG. 1, followed by the Y frames 320, the U frames 330, andthe V frames 340. Typical frames for the Y, U and V components areshown. In what follows we consider only the video stream part of thecontainer file, and without loss of generality (w.l.g.), we assume a YUVcolor space representation.

A camera may change its angle of view many times during the movie. Thesechanges of scenes, also called the movie's cuts, are distinguished bytheir shape and contents, see Ref. [3]. In terms of compression thismeans that we have little redundancy between the cuts.

The cut file 400 is schematically depicted in FIG. 4, comprising aheader section 410 followed by the cuts 420. The header is as follows:

n is the number of rows, m is the number of columns,N is the number of frames, and M is the number of cuts.

Each cut of the file has the same structure as the YUV file format givenin unit 300 of FIG. 3. For simplicity, we will proceed to consider fromnow on only one such component of each such cut. A generalization to allcomponents is straightforward.

Wavelets and multiwavelets, see Ref. [4], are important mathematicaltools that we use in the applications that follow. Classical discretewavelet transform (DWT) filters are depicted in FIG. 5; a pair of lowpass and high pass analysis filters are depicted in unit 510, and a pairof low pass and high pass synthesis filters are depicted in unit 520.For example, the one dimensional Haar transform is depicted in unit 530.

In general, we have m>1 filters, as depicted in FIG. 6; the analysisfilters are depicted in unit 610, and the synthesis filters in unit 620.For example, a 2D Haar transform is depicted in unit 630. Moregenerally, the filters may refer to the discrete multiwavelet transform(DMWT).

The lattice of integers ?^(n) is the set of n-tuples of integers in theEuclidean space ?^(n). A frame can be represented as a rectangular gridon the lattice ?², and a video as a cubic grid on ?³. A subset of alattice, which is itself a lattice is called a sub-lattice. Examples ofsub-lattices of ?² are given in FIG. 7. The Quincunx sub-lattices aredepicted in unit 710. The white circled points correspond to the evensub-lattice, and the dark circled points to the odd sub-lattice. TheDyadic sub-lattices are similarly depicted in unit 720. The Quincunxsub-lattices are determined by the dilation matrix of unit 715, and theDyadic sub-lattices by the dilation matrix of unit 725. The number ofsub-lattices is determined by the determinant of the correspondingdilation matrix, 2 in the Quincunx case, and 4 in the Dyadic case.Down-sampling refers to the process of extracting a sub-lattice from agiven lattice. For example, we display a dyadic down sampling in FIG. 8.The input signal is given in unit 810, a temporal down sampling in unit820, a spatial down sampling in unit 830, and a combined spatial andtemporal down sampling in unit 840.

SUMMARY OF THE INVENTION

According to an aspect of the present invention there is provided amethod of encoding video, comprising: receiving a video; performing ashrink operation on said received video, said shrink operationcomprising: creating a first lower resolution video from said video; andsequentially creating additional N−1 lower resolution videos, each oneof said additional lower resolution videos created from said precedinglower resolution video; compressing the lowest resolution video;creating a lowest resolution reconstructed video by decompressing saidlowest resolution compressed video; performing a first raise operationon said lowest resolution reconstructed video, said first raiseoperation comprising sequentially creating N higher resolutionsreconstructed videos, each one of said higher resolution reconstructedvideos created from said preceding lower resolution reconstructed videoby: creating a higher resolution video from said lower resolutionreconstructed video; computing a residual between said respective lowerresolution video and said created higher resolution video; compressingsaid computed residual; decompressing said compressed residual; andcombining said decompressed residual with said created higher resolutionvideo, yielding the respective higher resolution reconstructed video;and providing a bit stream comprising said lowest resolution compressedvideo, said compressed residuals and control information comprising saidN.

The bit stream may comprise low pass analysis filters and whereincreating a lower resolution video comprises applying a low pass analysisfilter to a video.

The bit stream may comprise blurring and down sampling operators andwherein creating a lower resolution video comprises applying a blurringoperator to a video and applying a down sampling operator to the bluroperation result.

The bit stream may comprise blurring and down sampling operators andwherein said low pass analysis filters are computed from said blurringand down sampling operators.

The bit stream may comprise low pass synthesis filters and whereincreating a higher resolution video comprises applying a low passsynthesis filter to a lower resolution reconstructed video.

The bit stream may comprise up sampling, interpolation, oracle anddeblurring operators and wherein creating a higher resolution videocomprises: a. applying an up sampling operator followed by aninterpolation operator to a lower resolution reconstructed video; b.applying an oracle operator to the interpolation operation result; andc. applying a deblurring operator to the oracle operation result.

The bit stream may comprise up sampling, interpolation, oracle anddeblurring operators and the low pass synthesis filters may be computedfrom said up sampling, interpolation, oracle and deblurring operators.

Computing a residual may comprise calculating the difference between therespective lower resolution video and the higher resolution video.

The bit stream may comprise high pass analysis filters and computing theresiduals may additionally comprise applying high pass analysis filtersto the calculated difference.

The high pass analysis filters may be computed from said calculateddifference.

Computing the higher resolution reconstructed video may comprise addingthe decompressed residual to the higher resolution video.

The bit stream may comprise high pass synthesis filters and creating thehigher resolution reconstructed video may comprise applying the highpass synthesis filters to the respective decompressed residuals andadding the results to the higher resolution video.

The high pass synthesis filters may be computed from low pass and highpass analysis filters and low pass synthesis filters.

According to another aspect of the present invention there is provided amethod of decoding a video encoded according to the above method,comprising: receiving and processing said bit stream comprising saidlowest resolution compressed video, said compressed residuals and saidcontrol information; creating a lowest resolution reconstructed video bydecompressing said lowest resolution compressed video; and performing asecond raise operation on said lowest resolution reconstructed video,said second raise operation comprising sequentially creating N higherresolutions reconstructed videos, each one of said higher resolutionreconstructed videos created from said preceding lower resolutionreconstructed video by: creating a higher resolution video from saidlower resolution reconstructed video; decoding the residual between saidrespective lower resolution video and said created higher resolutionvideo; and combining said decoded residual with said created higherresolution video, yielding the respective higher resolutionreconstructed video.

The bit stream may comprise low pass synthesis filters and creating ahigher resolution video may comprise applying a low pass synthesisfilter to a lower resolution reconstructed video.

The bit stream may comprise up sampling, interpolation, oracle anddeblurring operators and creating a higher resolution video maycomprises: a. applying an up sampling operator followed by aninterpolation operator to a lower resolution reconstructed video; b.applying an oracle operator to the interpolation operation result; andc. applying a deblurring operator to the oracle operation result.

The bit stream may comprise up sampling, interpolation, oracle anddeblurring operators and the low pass synthesis filters may be computedfrom said up sampling, interpolation, oracle and deblurring operators.

Computing the higher resolution reconstructed video may comprise addingthe decompressed residual to the higher resolution video.

The bit stream may comprise high pass synthesis filters and creating thehigher resolution reconstructed video may comprise applying the highpass synthesis filters to the respective decompressed residuals andadding the results to the higher resolution video.

The high pass synthesis filters may be computed from low pass and highpass analysis filters and low pass synthesis filters.

According to another aspect of the present invention there is provided avideo codec comprising: an encoding unit configured to encode a video; areconstructing unit configured to reconstruct said encoded video, thereconstructing unit creating a bit stream of decoding data; and adecoding unit configured to receive the bit stream of decoding data anddecode the video therewith.

The encoding unit may be configured to: receive a video; perform ashrink operation on said received video, said shrink operationcomprising: creating a first lower resolution video from said video;sequentially creating additional N−1 lower resolution videos, each oneof said additional lower resolution videos created from said precedinglower resolution video; and compressing the lowest resolution video.

The reconstructing unit may be configured to: create a lowest resolutionreconstructed video by decompressing said lowest resolution compressedvideo;

perform a first raise operation on said lowest resolution reconstructedvideo, said first raise operation comprising sequentially creating Nhigher resolutions reconstructed videos, each one of said higherresolution reconstructed videos created from said preceding lowerresolution reconstructed video by: creating a higher resolution videofrom said lower resolution reconstructed video; computing a residualbetween said respective lower resolution video and said created higherresolution video; compressing said computed residual; decompressing saidcompressed residual; and combining said decompressed residual with saidcreated higher resolution video, yielding the respective higherresolution reconstructed video; wherein said bit stream comprises saidcompressed lowest resolution video, said compressed residuals andcontrol information comprising said N.

The decoding unit may be configured to: receive and process said bitstream comprising said lowest resolution compressed video, saidcompressed residuals and said control information; create a lowestresolution reconstructed video by decompressing said lowest resolutioncompressed video; and perform a second raise operation on said lowestresolution reconstructed video, said second raise operation comprisingsequentially creating N higher resolutions reconstructed videos, eachone of said higher resolution reconstructed videos created from saidpreceding lower resolution reconstructed video by: creating a higherresolution video from said lower resolution reconstructed video;decoding the residual between said respective lower resolution video andsaid created higher resolution video; and combining said decodedresidual with said created higher resolution video, yielding therespective higher resolution reconstructed video.

According to another aspect of the present invention there is provided amethod of decoding a video encoded according to the method above,wherein said decoding unit is configured to analyze said reconstructedhigher resolution videos.

The decoding unit may be configured to compute the motion field of saidreconstructed videos.

The decoding unit may configured to perform object recognition of saidreconstructed videos.

BRIEF DESCRIPTION OF THE DRAWINGS

For better understanding of the invention and to show how the same maybe carried into effect, reference will now be made, purely by way ofexample, to the accompanying drawings.

With specific reference now to the drawings in detail, it is stressedthat the particulars shown are by way of example and for purposes ofillustrative discussion of the preferred embodiments of the presentinvention only, and are presented in the cause of providing what isbelieved to be the most useful and readily understood description of theprinciples and conceptual aspects of the invention. In this regard, noattempt is made to show structural details of the invention in moredetail than is necessary for a fundamental understanding of theinvention, the description taken with the drawings making apparent tothose skilled in the art how the several forms of the invention may beembodied in practice. In the accompanying drawings:

FIG. 1 describes the raw RGB file;

FIG. 2 describes the transforms between the RGB and YUV formats;

FIG. 3 describes the raw YUV file;

FIG. 4 describes the cut file;

FIG. 5 describes the 2-way Discrete Wavelet Transform Filters;

FIG. 6 describes the m-way Discrete Wavelet Transform Filters;

FIG. 7 depicts Lattices and sub-lattices;

FIG. 8 describes Dyadic Down Sampling;

FIG. 9 is a flowchart of the new Codec;

FIG. 10 is a flowchart of the Encoder;

FIG. 11 depicts the Bit Stream;

FIG. 12 is a flowchart of the Decoder;

FIG. 13 describes the M Codec;

FIG. 14 describes the M Codec;

FIG. 15 describes the M Codec;

FIG. 16 describes the O Codec parameters;

FIG. 17 describes the O Codec;

FIG. 18 describes the O Codec;

FIG. 19 describes the OM Codec; and

FIG. 20 describes the OM Codec.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides a new technology for video compressionthat unlike the standard MPEG methods such as H.264, benefits from theanalysis of the video in both the Encoder and the Decoder, using stateof the art computer vision and mathematical methods. For example, motionfield calculation and object recognition, see Ref. [1] and [7], may beused to reconstruct the video in both the Encoder and the Decoder.

A schematic diagram of the new generic Codec is depicted in FIG. 9. TheCodec consists of two main parts: The Encoder and the Decoder. TheEncoder 1000 compresses the video Y into the Bit Stream 1100, and theDecoder 1200 decompresses the Bit Stream into the reconstructed video Ŷ.The Bit Stream 1100, which is the output from the Encoder and the inputto the Decoder, represents the compressed video. The Bit Stream can bestored on disk or transmitted over a network. Both Y and Ŷ are referredto as “video” throughout the following description. Note that Y is acomponent of a cut as explained above.

In what follows we describe the Encoder 1000 see FIG. 10, the Bit Stream1100 see FIG. 11, and the Decoder 1200, see FIG. 12.

The Encoder 1000 has three stages as follows:

Stage I (E) consists of N iterations as follows:

Let us define Y₀ ? Y. Then video Y_(k) is the input to iteration k=0, .. . , N−1, and video Y_(k+1) is the output of iteration k=0, . . . ,N−1. Here, video Y_(k+1) is a lower resolution, coarser representation,of video Y_(k). Lower resolution meaning either spatial lowerresolution, temporal lower resolution, or both spatial and temporallower resolutions. We call this operation of lowering the videoresolution the Shrink operation. The number of such iterations, namelyN, is determined by the Encoder.

Stage II (E) consists of the Compress and Decompress operations. TheCompress operation refers to any method for compressing video asdiscussed for example in Pat. [1]. The Decompress operation refers tothe opposite operation of reconstructing the original video from thecompressed video. Let Y_(N) denote the resulting video after Stage I(E), then we denote the compressed video by F_(N), and the decompressedvideo by Ŷ_(N). Note that video Ŷ_(N) is a reconstruction of the videoY_(N) hampered by the quality of the compression method. In case of alossless compression, the reconstructed video Ŷ_(N) would be exactly thesame as Y_(N). However, in practice, the compression process is lossy,and Ŷ_(N) is not the same as Y_(N). Usually, the stronger thecompression is, the more the videos differ, and the more distortionexists. A good compression method keeps the viewing quality of thereconstructed video while minimizing the size of the compressed video.Note further that the compressed video F_(N) gets included in the BitStream 1100.

Stage III (E) consists of a first raise operation comprising N backwarditerations as follows:

In iteration k=N−1, . . . , 0, the reconstructed video Ŷ_(k+1) from theprevious iteration (or stage in case of k=N−1), is the input, and theoutput is the reconstructed video Ŷ_(k), which is of the same resolution(spatial and temporal) as video Y_(k). Hence, the final video Ŷ₀ is thereconstruction of the initial video Y₀, see Stage I (E) above.

In order to obtain Ŷ_(k) we do the following steps:

1) First, we resize video Ŷ_(k+1) into a higher resolution video Ĉ_(k),of the same resolution as that of video Y_(k). We call this operation ofraising the video resolution the Expand operation.

2) Next, we compute a residual R_(k) between video Y_(k) and theexpanded video Ĉ_(k). We call this operation of computing the residualthe Resultant operation.

3) Finally, we compress the residual R_(k) into E_(k). Here, again, bycompression, we mean any valid compression method as was discussed inStage II (E) above. We call this operation of compressing the residualthe Encode operation. Note further that the compressed residual E_(k)gets included into the Bit Stream 1100.

The next steps of the Encoder simulate the operations done at theDecoder:

4) We decompress the compressed residual E_(k) into the reconstructedresidual {circumflex over (R)}_(k). We call this operation ofdecompressing the compressed residual the Decode operation.

5) We combine the expanded video Ĉ_(k) with the reconstructed residual{circumflex over (R)}_(k) to get the reconstructed video Ŷ_(k). We callthis operation of combining the expanded video with the reconstructedresidual the Assemble operation. The Assemble operation is in a sensethe reverse of the Resultant operation.

The Bit Stream 1100 contains the following components:

The ctrl, the additional information known to the Encoder but unknown atthe Decoder, which is needed in order to decompress the Bit Stream. Forexample, the number of iterations N, which is determined by the Encoderis part of the ctrl. Note further that ctrl is transmitted in acompressed form.

The compressed video F_(N) .

The compressed residuals: E_(k), k=N−1, . . . , 0.

The Decoder 1200 has two stages as follows:

Stage I (D) consists of the Process and Decompress operations. The

Process operation retrieves the ctrl information and sets the necessaryparameters, such as for example the number of iterations N, required fordecoding. The Decompress operation decompresses F_(N) into the videoŶ_(N), as is done in Stage II (E) of the Encoder.

Stage II (D) consists of a second raise operation comprising N backwarditerations as follows:

In iteration k=N−1, . . . , 0, the reconstructed video Ŷ_(k+1) from theprevious iteration (or stage in case of k=N−1), is the input, and theoutput is the reconstructed video Ŷ_(k). This is done in the followingway (see also Stage III (E) of the Encoder):

1) Use the Decode operation to decompress the compressed residual E_(k)into the reconstructed residual {circumflex over (R)}_(k).

2) Use the Assemble operation to combine the expanded video Ĉ_(k) withthe reconstructed residual {circumflex over (R)}_(k) to get thereconstructed video Ŷ_(k).

To further clarify the invention, we describe hereby some possibleimplementations of the generic Codec. These are the M Codec, the 0Codec, and the OM Codec. Note, however, that many other implementationsare also possible and are not excluded by these examples.

EXAMPLE I The Multiwavelet (M) Codec

The M Codec is governed by a respective set of m_(k) multiwaveletfilters, so called the DMWT filters, as depicted in unit 1310 of FIG.13. Their role is explained in the following.

In the Encoder 1000:

Stage I (E) At iteration k=0, . . . , N−1, the Shrink operation isdetermined by the low pass analysis filter A₀ ^((k)). That is, we applyfilter A₀ ^((k)) to Y_(k) to get Y_(k+1), see unit 1320 of FIG. 13.

Stage 11 (E) is general as before.

Stage III (E) At iteration k=N−1, . . . , 0:

1) The Expand operation is determined by the low pass synthesis filterS₀ ^((k)). That is, we apply filter S₀ ^((k)) to Ŷ_(k+1) to get Ĉ_(k),see unit 1330 of FIG. 13.

2) The Resultant operation is determined by the m_(k) ? 1 high passanalysis filters A₁ ^((k)), A₂ ^((k)), ?, A_(m) _(k) ⁻ ^((k)):

First the difference between Y_(k) and Ĉ_(k), namely D_(k), is computed,see unit 1410 of FIG. 14.

Then, for j=1, . . . , m_(k) ? 1, we apply A_(j) ^((k)) to D_(k) to getthe respective residual component R_(k) ^((j)), see unit 1420.

3) We Encode the residuals R_(k) ^((j)) to E_(k) ^((j)) for j=1, . . . ,m_(k) ? 1, see unit 1430.

4) We Decode the reconstructed residuals {circumflex over (R)}_(k)^((j)) from E_(k) ^((j)), for j=1, . . . , m_(k) ? 1, see unit 1510 ofFIG. 15.

5) The Assemble operation is determined by the m_(k) ? 1 high passsynthesis filters: S₁ ^((k)), S₂ ^((k)), ?, S_(m) _(k) ⁻¹ ^((k)):

For j=1, . . . , m_(k) ? 1, we apply S_(j) ^((k)) to {circumflex over(R)}_(k) ^((j)) to get component {circumflex over (D)}_(k) ^((j)), seeunit 1520.

Then we reconstruct Ŷ_(k) by summing up:

Ŷ_(k)=Ĉ_(k) 30 Σ_(j=1) ^(m) ^(k) ⁻¹{circumflex over (D)}_(k) ^((j)) seeunit 1530.

In the Bit Stream 1100:

the ctrl information contains the DMWT filters in addition to N.

In the Decoder 1200:

Stage I (D) consists of the Process and Decompress operations, asbefore.

Stage II (D) At iteration k=N−1, . . . , 0:

1) We Decode the reconstructed residuals {circumflex over (R)}_(k)^((j)) from E_(k) ^((j)), for j=1, . . . , m_(k) ? 1, see unit 1510.

2) We use the Assemble operation as in Stage III (E) above toreconstruct video Ŷ_(k). That is, as in step 5 above, we apply S_(j)^((k)) to {circumflex over (R)}_(k) ^((j)) to get components {circumflexover (D)}_(k) ^((j)), and sum up these components with Ĉ_(k) to getvideo Ŷ_(k) , see units 1520 and 1530.

EXAMPLE II The Oracle (O) Codec

The O Codec is governed by a respective set of parameters, see FIG. 16.The set includes blurring/deblurring inverse operators, see unit 1610,down sampling/up sampling opposite operators and interpolationoperators, see unit 1620, and oracle operators, see unit 1630. Theirrole is described in the following.

In the Encoder 1000:

Stage I (E) At iteration k=0, . . . , N−1:

The Shrink operation is determined by the respective blurring and downsampling operators. That is, we apply the blurring operator B^((k)) toY_(k) yielding Z_(k), and then the down sampling operator ?^((k)) toZ_(k), yielding Y_(k+1), see unit 1710 of FIG. 17.

Stage 11 (E) is general as before.

Stage III (E) At iteration k=N−1 , . . . , 0:

1) The Expand operation is determined by the respective up sampling,interpolation, oracle and deblurring operators. That is, we apply the upsampling operator ?^((k)) followed by the interpolation operator I^((k))to Ŷ_(k+1) yielding video Z_(k) ⁰. Then we apply the oracle operatorO^((k)) to {circumflex over (Z)}_(k) ⁰, yielding {circumflex over(Z)}_(k), the reconstructed version of video Z_(k) of Stage I (E) above.The oracle operation, which reconstructs the best approximation{circumflex over (Z)}_(k) to Z_(k) based on {circumflex over (Z)}_(k) ⁰may use such methods as super resolution see Ref. [5] and compressedsensing see Ref. [6]. Finally, we apply the deblurring operator B_(inv)^((k)) to {circumflex over (Z)}_(k) yielding Ĉ_(k), see unit 1720.

2) The Resultant operation is simply the difference operation. That is,the residual R_(k) is the difference between Y_(k) and Ĉ_(k). Then, weEncode the residual R_(k) into E_(k), see unit 1810 of FIG. 18.

3) We then Decode the reconstructed residuals {circumflex over (R)}_(k)from E_(k). The Assemble operation being simply the summation operation,we add {circumflex over (R)}_(k) to Ĉ_(k) to get video Ŷ_(k), see unit1820.

In the Bit Stream 1100:

The ctrl information contains the parameters as in FIG. 16, in additionto N.

In the Decoder 1200:

Stage I (D) consists of the Process and Decompress operations, asbefore.

Stage II (D) At iteration k=N−1, . . . , 0:

1) We use the Expand operation as in step 1 of Stage III (E) above toreconstruct video Ĉ_(k) from video Ŷ_(k+1), see unit 1720.

2) We use the Decode operation to get {circumflex over (R)}_(k) and thenthe Assemble operation as in step 3 of Stage III (E) above toreconstruct Ŷ_(k) from Ĉ_(k) and {circumflex over (R)}_(k), see unit1820.

EXAMPLE III The Oracle Multiwavelet (OM) Codec

We combine the Oracle Codec together with the Multiwavelet Codec asfollows:

We define the m_(k) multiwavelet filters, see unit 1310, based on the OCodec parameters, see FIG. 16. We call the resulting method the OMCodec.

We now describe how to obtain the DMWT filters given the O Codecparameters:

1) We define the low pass analysis filter A₀ ^((k)) so that theresulting Shrink method approximates the given O Codec Shrink method,see unit 1910 of FIG. 19.

2) We define the low pass synthesis filter S₀ ^((k)) so that theresulting Expand method approximates the given O Codec Expand method,see unit 1920.

3) We define the set of m_(k) ? 1 high pass analysis filters A₁ ^((k)),A₂ ^((k)), ?, A_(m) _(k) ⁻¹ ^((k)), to be the multiwavelet filters thatannihilates Ĉ_(k), see unit 1930.

4) We complete the set of DMWT filters, by setting S₁ ^((k)), ? , S_(m)_(k) ⁻¹ ^((k)). This we do using the mathematical theory of wavelets,see Ref. [4] and Pat.[2].

The ctrl information contains the parameters as in FIG. 16, and thecorresponding DMWT filters as in FIG. 19, in addition to N.

A flowchart of the OM Codec is depicted in FIG. 20. The Encoderflowchart is depicted in unit 2010, and the Decoder in unit 2020.

The following documents are referenced in the application and are allincorporated by reference herein.

Patents [1] Ilan Bar-On and Oleg Kostenko, A Method And A System ForWavelet Based Processing, WO/2008/081459.

[2] Ilan Bar-On, Method And Apparatus For A Multidimensional DiscreteMultiwavelet Transform, U.S. Pat. No. 8,331,708 B2, Dec. 11, 2012.

References [1] “Computer Vision, A Modern Approach”, D. Forsyth and J.Ponce, 2012.

[2] “ITU-R Recommendation BT. 709”,http://en.wikipedia.org/wiki/Rec._(—)709

[3] “Cut by Cut”, G. Chandler, 2012. [4] “Wavelets and Multiwavelets”,Fritz Keinert, 2004. [5] “Super-Resolution Imaging”, P. Milanfar, Sep.2010. [6] “Compressed Sensing, Theory and Applications”, Y. C. Eldar etal., June 2012.

[7] “Optical flow”, http://en.wikipedia.org/wiki/Optical_flow.

1. A method of encoding video, comprising: receiving a video; performinga shrink operation on said received video, said shrink operationcomprising: creating a first lower resolution video from said video; andsequentially creating additional N−1 lower resolution videos, each oneof said additional lower resolution videos created from said precedinglower resolution video; compressing the lowest resolution video;creating a lowest resolution reconstructed video by decompressing saidlowest resolution compressed video; performing a first raise operationon said lowest resolution reconstructed video, said first raiseoperation comprising sequentially creating N higher resolutionsreconstructed videos, each one of said higher resolution reconstructedvideos created from said preceding lower resolution reconstructed videoby: creating a higher resolution video from said lower resolutionreconstructed video; computing a residual between said respective lowerresolution video and said created higher resolution video; compressingsaid computed residual; decompressing said compressed residual; andcombining said decompressed residual with said created higher resolutionvideo, yielding the respective higher resolution reconstructed video;and providing a bit stream comprising said lowest resolution compressedvideo, said compressed residuals and control information comprising saidN.
 2. The method of claim 1, wherein said bit stream comprises low passanalysis filters and wherein creating a lower resolution video comprisesapplying a low pass analysis filter to a video.
 3. The method of claim1, wherein said bit stream comprises blurring and down samplingoperators and wherein creating a lower resolution video comprisesapplying a blurring operator to a video and applying a down samplingoperator to the blur operation result.
 4. The method of claim 2, whereinsaid bit stream comprises blurring and down sampling operators andwherein said low pass analysis filters are computed from said blurringand down sampling operators.
 5. The method of claim 1, wherein said bitstream comprises low pass synthesis filters and wherein creating ahigher resolution video comprises applying a low pass synthesis filterto a lower resolution reconstructed video.
 6. The method of claim 1,wherein said bit stream comprises up sampling, interpolation, oracle anddeblurring operators and wherein creating a higher resolution videocomprises: a. applying an up sampling operator followed by aninterpolation operator to a lower resolution reconstructed video; b.applying an oracle operator to the interpolation operation result; andc. applying a deblurring operator to the oracle operation result.
 7. Themethod of claim 5, wherein said bit stream comprises up sampling,interpolation, oracle and deblurring operators and wherein said low passsynthesis filters are computed from said up sampling, interpolation,oracle and deblurring operators.
 8. (canceled)
 9. The method of claim 1,wherein said bit stream comprises high pass analysis filters and whereincomputing the residuals comprises applying high pass analysis filters tothe difference between the respective lower resolution video and thehigher resolution video.
 10. (canceled)
 11. (canceled)
 12. The method ofclaim 1, wherein said bit stream comprises high pass synthesis filtersand wherein creating the higher resolution reconstructed video comprisesapplying the high pass synthesis filters to the respective decompressedresiduals and adding the results to the higher resolution video. 13.(canceled)
 14. A method of decoding a video encoded according to themethod of claim 1, comprising: receiving and processing said bit streamcomprising said lowest resolution compressed video, said compressedresiduals and said control information; creating a lowest resolutionreconstructed video by decompressing said lowest resolution compressedvideo; and performing a second raise operation on said lowest resolutionreconstructed video, said second raise operation comprising sequentiallycreating N higher resolutions reconstructed videos, each one of saidhigher resolution reconstructed videos created from said preceding lowerresolution reconstructed video by: creating a higher resolution videofrom said lower resolution reconstructed video; decoding the residualbetween said respective lower resolution video and said created higherresolution video; and combining said decoded residual with said createdhigher resolution video, yielding the respective higher resolutionreconstructed video.
 15. The method of claim 14, wherein said bit streamcomprises low pass synthesis filters and wherein creating a higherresolution video comprises applying a low pass synthesis filter to alower resolution reconstructed video.
 16. The method of claim 14,wherein said bit stream comprises up sampling, interpolation, oracle anddeblurring operators and wherein creating a higher resolution videocomprises: a. applying an up sampling operator followed by aninterpolation operator to a lower resolution reconstructed video; b.applying an oracle operator to the interpolation operation result; andc. applying a deblurring operator to the oracle operation result. 17.The method of claim 15, wherein said bit stream comprises up sampling,interpolation, oracle and deblurring operators and wherein said low passsynthesis filters are computed from said up sampling, interpolation,oracle and deblurring operators.
 18. (canceled)
 19. The method of claim14, wherein said bit stream comprises high pass synthesis filters andwherein creating the higher resolution reconstructed video comprisesapplying the high pass synthesis filters to the respective decompressedresiduals and adding the results to the higher resolution video. 20.(canceled)
 21. (canceled)
 22. A video codec comprising: an encoding unitconfigured to encode a video; a reconstructing unit configured toreconstruct said encoded video, the reconstructing unit creating a bitstream of decoding data and a decoding unit configured to receive thebit stream of decoding data and decode the video therewith; wherein theencoding unit is configured to: receive a video; perform a shrinkoperation on said received video, said shrink operation comprising:creating a first lower resolution video from said video; sequentiallycreating additional N−1 lower resolution videos, each one of saidadditional lower resolution videos created from said preceding lowerresolution video; and compressing the lowest resolution video.
 23. Thevideo codec of claim 22, wherein the reconstructing unit is configuredto: create a lowest resolution reconstructed video by decompressing saidlowest resolution compressed video; perform a first raise operation onsaid lowest resolution reconstructed video, said first raise operationcomprising sequentially creating N higher resolutions reconstructedvideos, each one of said higher resolution reconstructed videos createdfrom said preceding lower resolution reconstructed video by: creating ahigher resolution video from said lower resolution reconstructed video;computing a residual between said respective lower resolution video andsaid created higher resolution video; compressing said computedresidual; decompressing said compressed residual; and combining saiddecompressed residual with said created higher resolution video,yielding the respective higher resolution reconstructed video; whereinsaid bit stream comprises said compressed lowest resolution video, saidcompressed residuals and control information comprising said N.
 24. Thevideo codec of claim 23, wherein said decoding unit is configured to:receive and process said bit stream comprising said lowest resolutioncompressed video, said compressed residuals and said controlinformation; create a lowest resolution reconstructed video bydecompressing said lowest resolution compressed video; and perform asecond raise operation on said lowest resolution reconstructed video,said second raise operation comprising sequentially creating N higherresolutions reconstructed videos, each one of said higher resolutionreconstructed videos created from said preceding lower resolutionreconstructed video by: creating a higher resolution video from saidlower resolution reconstructed video; decoding the residual between saidrespective lower resolution video and said created higher resolutionvideo; and combining said decoded residual with said created higherresolution video, yielding the respective higher resolutionreconstructed video.
 25. (canceled)
 26. (canceled)
 27. (canceled)